Recurrent Neural Network Training with Dark Knowledge Transfer

Recurrent neural networks (RNNs), particularly long short-term memory (LSTM), have gained much attention in automatic speech recognition (ASR). Although some successful stories have been reported, training RNNs remains highly challenging, especially with limited training data. Recent research found that a well-trained model can be used as a teacher to train other child models, by using the predictions generated by the teacher model as supervision. This knowledge transfer learning has been employed to train simple neural nets with a complex one, so that the final performance can reach a level that is infeasible to obtain by regular training. In this paper, we employ the knowledge transfer learning approach to train RNNs (precisely LSTM) using a deep neural network (DNN) model as the teacher. This is different from most of the existing research on knowledge transfer learning, since the teacher (DNN) is assumed to be weaker than the child (RNN); however, our experiments on an ASR task showed that it works fairly well: without applying any tricks on the learning scheme, this approach can train RNNs successfully even with limited training data.Comment: ICASSP 201

Room-Temperature Sputtered SnO2 as Robust Electron Transport Layer for Air-Stable and Efficient Perovskite Solar Cells on Rigid and Flexible Substrates.

Extraordinary photovoltaic performance and intriguing optoelectronic properties of perovskite solar cells (PSCs) have aroused enormous interest from both academic research and photovoltaic (PV) industry. In order to bring PSC technology from laboratory to market, material stability, device flexibility, and scalability are important issues to address for vast production. Nevertheless, PSCs are still primarily prepared by solution methods which limit film scalability, while high-temperature processing of metal oxide electron transport layer (ETL) makes PSCs costly and incompatible with flexible substrates. Here, we demonstrate rarely-reported room-temperature radio frequency (RF) sputtered SnO2 as a promising ETL with suitable band structure, high transmittance, and excellent stability to replace its solution-processed counterpart. Power conversion efficiencies (PCEs) of 12.82% and 5.88% have been achieved on rigid glass substrate and flexible PEN substrate respectively. The former device retained 93% of its initial PCE after 192-hour exposure in dry air while the latter device maintained over 90% of its initial PCE after 100 consecutive bending cycles. The result is a solid stepping stone toward future PSC all-vapor-deposition fabrication which is being widely used in the PV industry now

Focused image search in the social Web.

Recently, social multimedia-sharing websites, which allow users to upload, annotate, and share online photo or video collections, have become increasingly popular. The user tags or annotations constitute the new multimedia meta-data . We present an image search system that exploits both image textual and visual information. First, we use focused crawling and DOM Tree based web data extraction methods to extract image textual features from social networking image collections. Second, we propose the concept of visual words to handle the image\u27s visual content for fast indexing and searching. We also develop several user friendly search options to allow users to query the index using words and image feature descriptions (visual words). The developed image search system tries to bridge the gap between the scalable industrial image search engines, which are based on keyword search, and the slower content based image retrieval systems developed mostly in the academic field and designed to search based on image content only. We have implemented a working prototype by crawling and indexing over 16,056 images from flickr.com, one of the most popular image sharing websites. Our experimental results on a working prototype confirm the efficiency and effectiveness of the methods, that we proposed